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Documenting In-Classroom Support and Coaching Activities of a 
Professional Development Program Directed toward School-Wide Change: An 
Integral Part of an Organization’s Evaluation Efforts 

Kathryn Race 
Evelyn Ho 
Leah Bower 1 
Evaluation Department 

Teachers Academy for Mathematics and Science 
Chicago, 1L 

This study examines the classroom practices of 265 elementary school teachers from 
the Chicago Public School system who were participating in an intensive 
professional development program. The program included classroom coaching and 
mentoring activities in support of instructional sessions conducted over time. Based 
on over 1 ,700 standardized implementation logs, a descriptive assessment looked at 
changes in the level of intervention by professional development staff throughout the 
classroom support phase of the program. Evidence related to program outcomes that 
address the use of standards-based curricula in mathematics and science and the use 
of best practices in the classroom by participating teachers were examined as well. 
Also examined was the degree to which varied instructional strategies were used in 
the classroom in support of diverse learning styles of the children who ultimately 
should benefit from this program. Limitations of this methodology are discussed as 
well as the overall results which suggest the feasibility and utility of this 
methodology in recording hard-to-capture data. 

Posited by Kaser and Bourexis (1999), a key characteristic of a high-quality, profes- 
sional development program in mathematics and science is that the learning activities for 
teachers are spread over time. As will be described shortly, the Academy’s professional 
development program for elementary school teachers meets this criterion. In addition, a 
distinct feature of the Academy’s program is the extended support that is offered through 
scheduled classroom visits. These classroom visits involve coaching and reflective 
instructional interaction between teachers and professional development staff 
and provide the teacher with an opportunity to try out their new knowledge and skills 
while receiving timely support and feedback from staff who are familiar with the 
underlying program content and pedagogy associated with these lessons (Kaser & 
Bourexis, 1999). The coaching and reflective instructional support that is provided 
through these visits is summarized in a standarized implementation log. 

The program that will be described shortly is a professional development program for 
teachers. Although targeted toward teachers, the ultimate accountability of the program 
is likely to be evidenced by changes in the classroom experience of students as well as 

1 Now at Ramsey County Community Human Services Division, St. Paul, MN. 
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student achievement levels. The assessment of this program at the student level is 
beyond the scope of this work. Nonetheless, recent research suggests linking differential 
teacher effectiveness as a strong determinant of differences in student learning (Darling- 
Hammond, 1999), that quality teachers are a major influence on student achievement 
(Ferguson, 1991) and improved teachers’ education has a significant impact on 
improving student achievement (Greenwald, Hedges & Laine, 1 996). After brief 
background information, this paper will focus on the classroom support component of 
this program. We will then highlight what we have learned from the series of 
implementation logs that describe the classroom support and coaching activities, with 
particular attention given to the review of evidence of transference to the classrooms of 
students from participating schools during the first year of program intervention. 

Background 

The Teachers Academy for Mathematics and Science is a non-profit organization located 
in Chicago. The Academy offers an intensive 3-year professional development program 
designed to meet the needs of under-prepared elementary school teachers in Chicago and 
select school districts in Illinois (Brett, 1996). The program is directed toward high-risk 
schools, where the proportion of students that are not making grade-level standard in 
mathematics as well as science is high and evident across grades. The program recently 
underwent a major redesign effort to better serve the needs of its target audience. As 
revised, the program content is offered by instructional level (i.e., primary, intermediate 
and upper grade levels) that blends mathematics and science curricula with technology. 
The program is designed to provide 60 hours of professional development instruction per 
year for two years that is developmentally appropriate by grade-level, based on national 
and state standards in mathematics and science, content driven and inquiry based using 
nationally recognized curricula. 

In addition, 15 contacts occur, which involve coaching and reflective instructional 
support in the classroom per year during the first two years. The intent of these 
classroom visits is to facilitate the transfer of program content and pedagogy by the 
teacher in the classroom through modeled lessons by professional development staff, co- 
taught lessons, and observed lessons. This implementation support offered through the 
Academy program differs in at least one important feature from the clinical teaching 
experiences that are reported in the literature. The latter approach involves observing and 
providing feedback to teachers who are attempting new methods in an environment away 
from the classroom where they typically teach. Assessment of these clinical teaching 
experiences suggests, however, that these teachers have had difficulty applying these 
newly learned practices once back at their home schools (Miech, Nave & Mosteller, 
2001). In contrast, the Academy’s program component offers this support during visits 
conducted in the actual classroom where the teachers typically teach with reflective 
discussion and planning conferences happening before and/or after these visits. Also, 
during the course of these visits it is intended that this implementation support move from 
a high level of intervention in the classroom by professional development staff (by 
modeling a lesson(s) or co-teaching) to less involvement by the professional developer 
based on observation-only classroom visits. Thus, the potential for transfer of best 
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practices from program instruction to classroom teaching theoretically should be greater 
using this model. 

The program is also supported by distributed teaching materials, student manipulatives, 
and technology resources (Feranchak, Avichai, Langworthy & Triana, 2001). The 
overall program intervention occurs within the context of a school-wide systemic effort 
that requires that a high percent of mathematics and science teachers within each school 
participate in the program and includes program outreach to principals and school 
administrators as well as parents and community members. The third year provides a 
year of transition to help the school sustain progress after the program. 

Evaluation Framework 

The Academy has focused its evaluation efforts in four major areas: program design, 
program delivery, program outcomes, and the periodic assessment of the organization as 
a whole (Race, 2000). In keeping with recommendations by the National Center for 
Improving Science Education (NCISE), the Academy recognizes that the balance 
between process and outcome evaluation efforts is essential (Brett & Scheirer, 1994). 

The intent of the outcome-based evaluation component is to begin to build a framework 
that will facilitate continuity of data collection and analysis over time and foster the 
ability to make valid inferences about the effectiveness of the Academy’s professional 
development program (Brett & Scheirer, 1994). 

Accordingly, an internal task force comprised of Academy program, operations, and 
evaluation staff developed a detailed outcome map. The program is based on this 
cohesive set of intermediate, long-term, and ultimate outcomes, which form the basis of 
the program outcomes component of the evaluation framework. The map builds on 
intermediate outcomes that address teacher performance and practices along with 
additional intermediate outcomes that focus on the impact of the school and community 
environments. Long-term outcomes focus on teacher attitudes, knowledge, and practice, 
and ultimate outcomes address student academic performance and enthusiasm for 
mathematics and science. 

Purpose 

A critical teacher-related program outcome concerns the transfer of instructional practices 
in the classroom relative to using standards-based curricula in mathematics and science, 
the use of best practices, and offering a variety of lesson formats. The Academy has 
identified two program outcomes that focus on teacher instructional practices and are of 
immediate concern here: 

• Teachers use best practices in their instruction of mathematics, science, and 
integrated technology. 

• Teachers demonstrate increased ability to apply national and state 
mathematics, science, and technology standards to their teaching. 
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Through descriptive analysis of over 1,700 implementation visit logs, these program 
outcomes are assessed. This analysis is based on implementation visit logs from 
observations of one cohort of elementary school teachers during one year of instruction 
and implementation (from 16 participating schools from the Chicago Public School 
system). More specifically, the focus of this analysis is to highlight the instructional 
processes and practices of participating teachers throughout the school year based on 
evidence of: 

• Changes in level of intervention by the professional development (PD) staff 
throughout implementation support. 

• Content alignment with state standards in mathematics and science. 

• Use of best practices in the classroom. 

• Use of a variety of instructional strategies (i.e., varied lesson formats to meet 
the diverse learning styles of the students in participating schools). 

Method 



Overview of Implementation Support 

As noted, an integral part of the Academy program involves implementation support for 
participating teachers via 1 5 contacts per year. A contact consists of a combined 
classroom visit and a conference with the participating teacher (except for the first visit, 
which tends to involve either a conference or a classroom visit to start the process). This 
paired classroom visit-conference constitutes one implementation log as described below. 
The conferences allow for the teacher and professional developer (PD) to reflect and 
discuss classroom activities and issues as well as identify ways to improve future 
instruction. In addition, in-session instruction time is set aside for teachers to plan how 
Academy or Academy-modified instruction can be incorporated in their classrooms. 

Implementation visits begin shortly after the instructional portion of the program has 
started and continue throughout the school year until 15 contacts have been successfully 
completed or scheduled (approximately one combined classroom visit and conference a 
month per teacher). If a teacher received all planned implementation sessions, this would 
result in a total of either seven completed logs for that teacher; or eight completed logs if 
the first visit was an introductory or get-acquainted session. 

Implementation Logs 

The implementation log serves to document this program component. The log is 
completed after a classroom visit by professional development staff of the Academy and 
describes a classroom lesson through a standardized set of closed- and open-ended 
questions. These closed-end questions include: who took the teaching lead (i.e., the 
lesson was modeled by Academy professional development staff; co-taught by PD staff 
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and the teacher; or the teacher taught the lesson and the PD staff member only observed 
the lesson). Also queried was the content area covered in the lesson; whether the content 
area is aligned with state mathematics and science learning standards; the primary 
instructional strategy used during the lesson; observed lesson format(s); and, best 
practices evident during the lesson. A narrative portion of the log serves to elicit 
reflections from both the teacher and staff through several open-ended questions. These 
reflections focus on teacher progress, the next steps for implementation, and the 
discussion that occurred during the conference with the teacher. The gathering of this 
information was supported by a standardized protocol, which included as already stated a 
standardized form, instructions and operational definitions of terms, and a schedule for 
processing these forms. 

Cancelled visits are documented also in the implementation log. This documentation 
provides a brief explanation of the reason for the cancellation, and whether the visit had 
been rescheduled. If a pattern of missed visits occurred for a particular teacher it was 
discussed by the PD and supervisory program staff as needed. 

Procedure 



During the 1999-2000 school year, an electronic database was created and maintained by 
the technology staff of the Academy. The implementation log is housed electronically, 
within this database, which is supported by Filemaker Pro software. Following a 
standard procedure, each professional developer completed an implementation log after 
each classroom visit (or scheduled but cancelled visit) using their individual database. 
Based on a process schedule, these data were periodically fed into an Academy-wide 
implementation log database. The Academy-wide database houses the completed 
implementation logs compiled during a given school year for all teachers receiving 
implementation. For analysis purposes, quantitative data were exported from Filemaker 
Pro into SPSS (1999). 

Approach to Data Analysis 

For present purposes, data are summarized descriptively through a series of tables and 
figures. Focusing largely on the quantitative data taken from the implementation logs, 
the current analysis examined: 1) changes in level of intervention by the program 
development staff throughout the implementation phase of the program, 2) whether 
observed content areas align with state mathematics and science standards, 3) evidence of 
best practices, and 4) the use of varied instructional strategies in the classroom. 

To augment the quantitative assessment of these data, a small subset of implementation 
logs from participating primary and intermediate-level teachers were selected. That is, 
existing implementation visit logs from ten participating teachers from each instructional 
level were selected such that the sample included at least one set of logs for at least one 
teacher per professional development staff member. Thus a total of 159 implementation 
visit logs from 20 teachers were qualitatively reviewed. To this end, a hard copy of each 
selected implementation log summary was analyzed. General patterns were noted with 
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Table 1 

Summary of Completed, Cancelled and Total Number of Visits 
by Instructional Level 





Instructional Level 


Class Visit 


Primary 


Intermediate 




Yes 


No 


Total 


Yes 


No 


Total 


Log 1 


137 


21 


158 a 


101 


6 


107 


Log 2 


149 


5 


154 b 


93 


8 


101 b 


Log 3 


135 


14 


149 b 


82 


10 


92 b 


Log 4 


127 


14 


141 b 


81 


10 


91 b 


Log 5 


120 


11 


131 b 


76 


13 


89 b 


Log 6 


99 


24 


123 b 


81 


5 


86 b 


Log 7 


85 


24 


109 b 


64 


14 


78 b 


Log 8 


68 


23 


91 


43 


2 


45 b 


Total 


920 


136 


1056 


621 


68 


689 



Include 6 teachers that were not part of the implementation phase of the 
program. Three other teachers dropped out of the program after the first visit. 
Reflects a drop in the number of participating teachers. 



particular attention given to describing how the transition from PD-modeled lessons to 
teacher-taught lessons occurred. Additional analytic questions (e.g., missed visits, 
program attrition) were explored through qualitative review of appropriate subsets of 
implementation logs as well. 



Results 

A total of 265 teachers received documented implementation support during year one of 
the program (across schools, this resulted in 158 primary and 107 intermediate-level 
teachers). A total of 1,745 documented implementation logs were completed with 82% 
of the planned visits completed or scheduled (if all 265 participating teachers received 8 
visits); 1,541 logs represented successfully completed paired classroom-conference visits. 
Table 1 shows a breakdown of implementation visits by instructional level (primary or 
intermediate). As can be seen, a small but steady decrease in the total number of logs 
(documented visits) occurred from logs 2 through 6, with a larger drop off occurring for 
visits seven and eight. The decrease in number of logs across visits reflects a decrease in 
the number of participating teachers receiving implementation up to log seven. A log for 
the eighth visit was applicable only for those teachers whose first visit was an initial get- 
acquainted session. 

Missed Visits. Implementation logs that documented missed visits were analyzed 
qualitatively for general patterns as to why a lesson did not occur. In order of most 
frequently mentioned, three general patterns were noted. Most often, visits were not 
completed as planned because the teacher was absent (due, for example, to illness, 
family-related issues, or absent without apparent reason); or, the teacher cancelled 
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because they were not prepared for the lesson. Very often, this lack of preparation on the 
part of the teacher was also associated with not regularly attending the instructional 
sessions of the program. These teachers very often subsequently dropped out of both the 
instructional and implementation component of the program. A third general pattern 
involved scheduling problems or conflicts. Early in the program these scheduling 
problems often were associated with the teachers not fully understanding the require- 
ments of the program. Later these scheduling conflicts often centered on school- 
sponsored activities such as planned assemblies, standardized test practice or actual test 
taking, and field trips. 

Attrition. Implementation logs were examined to determine program-related reasons for 
teachers dropping out of the program before completion. Teacher attrition was evident as 
early as after the initial visit, but also well into the latter portion of implementation. The 
main reason teachers dropped out was the time commitment required to attend sessions, 
which conflicted with other after-school programs or perceived to interfere with 
standardized test preparation. Other teachers personally felt they did not find the 
professional development program useful, citing that the time and effort needed to attend 
the program did not outweigh its benefits. Specific reasons given were that the activities 
were not applicable, that they felt others were judging them, and that they were being 
forced to join the program. Other teachers transferred to a new school in the middle of 
implementation or scheduling difficulties between the professional developer and teacher 
also resulted in teacher attrition in a few cases. 

Changes in Level of Intervention across Classroom Visits 

During a classroom visit, the professional development (PD) staff of the Academy either 
modeled a lesson, co-taught a lesson with the teacher, or observed the teacher teaching a 
lesson. Figures 1 and 2 show a breakdown of this level of intervention across these 
classroom visits for both primary and intermediate grade-level classes. As can be seen, 
the number of PD-modeled lessons decreased across implementation visits, with a shift 
toward co-taught or PD-observed lessons occurring at or around log three. Despite the 
tendency for proportionally more intermediate lessons to be modeled by PD staff, the 
shift toward co-taught and teacher-taught lessons was evident for both primary and 
intermediate-level classroom visits. 

Qualitative Assessment. A qualitative review of the previously mentioned subset of 
implementation logs suggested that there may be many factors associated with the 
transition from a PD-modeled to teacher-taught lesson. Although not intended to imply 
causality, the following factors were frequently mentioned. These included regular 
attendance during the professional development sessions of the program by the 
participating teacher. Also for some teachers, the fact that a particular aspect of the 
program was already familiar to a teacher (e.g., curriculum, activities, or instructional 
strategies) seemed to ease him or her into taking the lead. Examples here included: the 
teachers' existing comfort level with students working in groups, having used other 
activities from related curricula; already being comfortable with asking open-ended 
questions; and, existing comfortable level with interacting with student activities by 
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Percent Percent 



Figure 1. PD/Teacher Role: Primary Program 
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Figure 2. PD/Teacher Role: Intermediate Program 
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actively jumping in during an activity to make a point, clarify an issue or to encourage 
reflections among the students; or their own existing level of enthusiasm or willingness to 
try new ideas. Often, the positive reaction of their students to a PD-model lesson served 
to act as a catalyst for teachers to attempt something on their own. Moreover, their own 
willingness to take a small step forward (such as re-arranging the classroom from rows of 
desks into small group work stations) often led a teacher to try other ideas when their 
initial efforts met with favorable responses by their students (e.g., moving from group 
work to cooperative learning tasks within groups). In some instances, a PD had to model 
several lessons before a more timid or reluctant teacher would take the lead and either co- 
teach or lead a lesson. For one very traditional teacher, it seemed that the content of the 
lesson (i.e., a science lesson on volume) was a point of interest and led to his/her taking 
the lead. 

Noted obstacles tended to involve classroom management issues for difficult to manage 
classes that required frequent disciplinary action. Class management issues were 
reduced, in some cases, by teachers using more tangible cues as guides (e.g., using a bell 
to signal to students that it was time for them to stop and listen) or to provide clearer 
instructions as to what was expected of the students during an activity. At other times, the 
teachers' concern regarding classroom management issues really stemmed from their 
reluctance to relinquish some control over the classroom to facilitate more cooperative 
and engaged learning among their students and to become comfortable with the noise 
level often associated with hands-on activities in the classroom. 

Other obstacles included lack of preparation time for teachers to plan a lesson and the 
teacher’s own limited content knowledge in either mathematics and/or science and the 
need to increase their knowledge and confidence before they would/could take the lead to 
teach a lesson in this content area. 

Instructional/Implementation Program Alignment with State Standards in Mathematics 
and Science 



The Academy program is designed to align with state (in this case Illinois) learning 
standards in mathematics and science (Illinois State Board of Education, 1997). 

Although it is intended that the program cover all of these standards, it is not intended 
that each standard be addressed equally across grade levels. As developmentally 
appropriate, some content areas are likely to receive more emphasis than others and this 
pattern should change from primary to intermediate-level instruction. Such an approach 
is consistent with recommendations made by the National Council of Teachers of 
Mathematics (NCTM, 2000). 

Figures 3 and 4 summarize the alignment of the instructional and implementation phases 
of the Academy program with state mathematics and science standards based on 
participating Chicago schools during the 1999-2000 school year. For the instructional 
program, these proportions are based on the number of program hours that cover a 
particular standard. By design, some lessons address both mathematics and science 
standards. Thus, if a lesson addressed both mathematics and science, an applicable 
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standard is noted in each discipline. (It should be recognized that the program staff of the 
Academy compiled these instructional program data.) For the implementation portion of 
the program, the number of sessions where a mathematics or science standard was 
observed was divided by the total number of observations. Some variability between the 
instructional portion of the program and the implementation portion of the program 
should be expected based on how the content area aligns with state standards. The larger 
question, however, is how much variability can there be (or should there be) and still be 
within the expected parameters suggested by the instructional program. To help gauge 
this decision we have used a nonparametric, goodness-of-fit test, the Kolmogorov- 
Smimov (K-S) one sample test (Siegel & Castellan, 1988) to assess the similarity of the 
instructional and implementation portions of the program. 

Comparing the instructional and implementation portion of the primary program, the 
apportionment of Mathematics Learning Standards was statistically significant, if 
marginally so [D( max ) = .049, p <05] based on the K-S goodness-of-fit test. This suggests 
that variations in lesson content observed during primary-level implementation support 
probably differed from what the instructional program would have suggested. In a 
similar manner, these comparisons were made for the intermediate-level program. Based 
on a goodness-of-fit test, these differences between instructional and implementation 
program components for year one of the intermediate-level program were statistically 
significant as well. This suggests that the content emphasized during the intermediate- 
level implementation was not entirely aligned with the content presented during the 
instructional program [D(max) = .087, p <.01 ]. Regarding Science Learning Standards, the 
primary-level implementation support does not show the expected alignment as suggested 
by the instructional component of the program [D( max )= .108,p <.01 ] and in a similar 
manner, the intermediate-level implementation component of the program also did not 
align with the instructional support relative to Science Learning Standards [D(max)= .14, 
P< OlJ- 

Figures 5 through 8 show a percent breakdown of the State Mathematics and Science 
Learning Standards across classroom visits for primary and intermediate grade levels. For 
these graphs, the pattern of standards-based content should be reviewed discretely within 
visits and not necessarily across visits. As can be seen in Figure 5, there are a mix of 
State Mathematics standards noted for observed primary lessons, with State Mathematics 
Goal 6: Number Sense/Operations, Goal 10: Data Analysis/ Probability, and Goal 7: 
Estimation and Measurement often marked across visits. State Mathematics Goals 8: 
Algebraic and Analytical Methods and State Goal 9: Geometric Methods were noted less 
often. A different pattern is evident for these same logs directed toward intermediate 
classrooms (see Figure 6), with a more blended mix across all of the State Mathematics 
Goals. 

Regarding State Science Standards for primary lessons (see Figure 7), it is clear that State 
Science Goal 1 1 : Process is most often mentioned from the early logs through log 5. 
Science Process and Content (State Goals 1 1 and 12) tend to balance out from the 6th 
through 8th logs. In Figure 8, the first two lessons for intermediate-level lessons (log 1 
and 2) reflect a blending of State Science Goals 1 1 through 13. These visits, it is 
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Figure 3 

State Mathematics Standards Addressed during the First Year of the Instructional Program 
and Implementation Participating Chicago Public Schools 1999-2000: 

Primary and Intermediate Programs 




Percent of Percent of 

Instructional Program Implementation Log 
Visits 



Intermediate 




Percent of Percent of 

Instructional Program Implementation Log 
Visits 



State Mathematics Standards 

Estate Goal 6: Number sense/operations Estate Goal 7: Estimation and measurement 

Estate Goal 8: Algebraic and analytical methods Estate Goal 9: Geometric methods 

Estate Goal 10: Data analysis/probability 



Figure 4 

State Science Standards Addressed during First Year of Instructional Program and Implementation 
Participating Chicago Public Schools 1999-2000: 

Primary and Intermediate Programs 




Visits Visits 



State Science Standards 

I □ State Goal 11: Inquiry and design □ State Goal 12: Concepts and principles □ State Goal 13: Science, technology and society 





Figure 5. State Mathematics Standards Addressed: Primary Program Figure 6. State Mathematics Standards Addressed: Intermediate Program 
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recalled, were most often PD-modeled instruction in the classroom. Thus, these may 
represent an ideal and not what the teachers are doing in practice. For remaining log 
visits, State Science Goal 1 1 : Process was most frequently marked, with evidence of 
State Science Goal 12: Content less frequently marked. State Science Goal 13: Science, 
Technology and Society was mentioned least frequently compared to the other science 
standards. It was more frequently observed, however, across visits involving 
intermediate grade levels as compared to observed primary lessons. 

Evidence of Best Practices Observed in the Classroom and Varied Instructional Strategies 

Tables 2 and 3 provide a summary of the frequency (based on percents) in which best 
practices were evident during observed primary and intermediate lessons, respectively. 
Tables 4 and 5 provide the same level of information for evidence of the use of varied 
lesson format styles again for primary and intermediate observed lessons. The most 
commonly observed best practices and instructional strategies in each log (column) are in 
bold face type. Across the top of the tables are labeled the most common level of 
intervention used for these lessons (e.g., PD-led, co-taught). 

Of the nine possible best practices listed in Table 2, Hands-on/Minds-on Approach was 
mentioned most frequently across visits involving primary-level classrooms. The use of 
this best practice was similar when level of intervention was taken into consideration. 
That is, a high percent of co-taught and observed primary-level lessons often involved the 
best practice of Hands-on/Minds-on Approach. To a lesser extent, the use of the best 
practice of Addressing a Variety of Learning Styles, Higher-order Thinking Skills, and 
Constructivism were also observed. As many as 38% of the lessons that occurred during 
the first classroom visit, which were primarily teacher-taught lessons, incorporated No 
best practices. This percent decreased to a range of 1 to 9% as implementation support 
progressed. 

For intermediate-level classrooms (see Table 3), the best practice of Hands-on/Minds-on 
Approach was frequently observed. When level of intervention was taken into account, 
this best practice was still observed at a high percent throughout log visits in both co- 
taught and observed intermediate-level lessons. To a modest extent, there was evidence 
of the use of Addressing a Variety of Learning Styles, Higher-order Thinking Skills; 
Real-world Connections and Assessment/Feedback was evident during select classroom 
visits. 

Regarding instructional strategies for primary-level classroom visits (see Table 4), a 
variety of lesson format styles were observed with Activity observed most often. This 
high percent of lessons involving Activity is also evident in primary-level lessons that 
were co-taught or observed. To a lesser extent, the lesson formats of Discussion, 
Teacher-directed, Group Work, and Experiment/Investigation were noted. Regarding 
instructional strategies for intermediate classroom visits (see Table 5), instructional 
strategies involving Discussion was frequently observed. Experimentation/Investigation 
was observed frequently during co-taught and teacher-taught lessons. There is modest 
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Table 3 

Best Practices by Professional Development Staff-Teacher Role: Intermediate Level 
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Table 4 

Lesson Formats by Professional Development Staff-Teacher Role: Primary Level 




-C 

D> 

3 

(0 

*-* 

i 

o 

o 



*D 

<D 

X 






Log Number 8 
n=91 h 
(Percent) 


r*“ 

d 

CD 


o> 

d 

co 


co 

CD 


CNJ 

CD 


a» 

td 


05 

d 

CO 


CM 

CD 

T— 


to 

CO 

CM 


1 23.51 


M - 




Log Number 7 
n=109 9 
(Percent) 


IO 

CO 

co 


co 

co 


CO 

to 

to 


CO 

id 




r^ 

CM 


CD 

S-' 


to 

CD 


05 

td 

CM 


CM 

00 


CM 

T*“ 


Log Number 6 
n=123* 
(Percent) 


CO 


co 

co 

CO 


to 

T*“ 

to 


CM 

CO 

CM 


o 

CO 


CO 

oo 

CM 


CM 


CM 

CD 


CO 

d 

CM 


T— 

o 


© 

CM 


Log Number 5 
n=131 e 
(Percent) 


CM 

d 

1^ 


CO 

co 

CM 


h*- 

CD 


CO 

00 


CD 


00 

d 

CM 


CD 


I s - 


CO 

CO 

CM 


CO 

oo 


to 

CM 


Log Number 4 
n=141 d 
(Percent) 


o 

co 

I s ** 


CM 


CO 

CO 


r— 

co 


o 


CD 

CO 

CM 


r-- 

id 


CD 

CM 


r>- 

id 


t— 


h- 


Log Number 3 
n=149 c 
(Percent) 


o> 
i d 
CO 


CO 

d 

T“ 


Ti; 

d 

to 


CD 

CM 


o 

co 


CO 

CO 


r— 


CD 


h*. 

CD 

CM 


M; 

mt 


CM 

td 


Log Number 2 
n=154 b 
(Percent) 


to 

Tf 

h* 


T*“ 

Tf 

T“ 


CO 

CO 

CD 


O 

M- 


CM 


CO 


r— 

d 


o 

CD 


O 

d 


CO 


Tf 

CO 


Log Number 1 
n=158 a 
(Percent) 


IO 

n 


CM 

d 

CM 


CM 


CO 

CM 


05 

CO 


CD 

CD 


00 

00 


00 

00 


CD 

CD 


CD 

CO 


CM 


Lesson Format 


> 

o 

< 


iTeacher directed 


1 Discussion I 


E 

c 

3 

C 

0 

-o 

c 

0 

Q 

0 
~o 
c 

XL 

1 

"co 

0 

CO 


C/5 

c 

o 

5 

c 

0) 

C/5 

0 

a 

c 

0 

■D 

3 

CO 


1 Group work 


a 

c 

1 

0 

> 

0 

ac 


ITeacher lecture/demo 


C 

o 

g 

w 

0 

> 

C 

d 

c 

05 

| 

0 

Q 

X 

LU 


1 Games/contests 


k. 

© 

.c 

s 











co 










0 










CO 










0 










O 


CO 

0 




CO 


CO 


O 

CM 


C/5 


C/5 


0 


0 


0 


0 


CO 


CO 




o 


CO 


0 


0 


C 




0 


O 


o 


o 


CO 




in 


r^ 


■Q 


T— 


05 


co 


CM 


0 




mt 


T— 


T— 


CO 


o 


t_ 


c 


c 


0 


TD 

0 


c 

o 


O 

TD 


o 

TD 


-Q 

CO 


C/5 


TD 


0 


0 


c 


CD 


0 


C/5 


CO 


0 


_Q 


C/5 


0 


0 


o 


C/5 

C 


0 

JD 

C/5 


JD 

C/5 


n 

CO 


0 

CL 


0 


C 


c 




o 


C 


0 


0 


c 


k_ 


0 


o 


o 


0 


0 


o 


k_ 




-c 


Cl 


k_ 


0 


0 






0 


CL 


CL 


O 




CL 








0 










-C 


C 


0 


0 




O 


0 


£ 


£ 


TD 

0 






o 


O 


■o 


o 


o 


o 


o 


■o 

0 


TD 


o 






0 




TD 


TD 


0 


"O 


td 


0 


0 


TO 


TD 


0 


TD 


■o 


*o 


0 


"O 


TD 


•o 


05 

C 

CO 


0 

0 


TD 

0 

0 


0 

0 


0 

0 


*o 


0 


0 


CO 


05 

c 


0 

TD 


*o 

05 


TD 

03 


E 


to 

CO 


05 


C 


C 


05 


c 


CO 


CO 


c 


E 


C/5 


CO 


g 


CO 


C/5 


d 


d 


CO 



CO 

0 


> 


> 

CO 


> 

CO 




1 

CO 


> 


5 

CO 


CO 


CO 


0 


0 


0 


0 


0 


0 


0 


0 


CO 


CO 


CO 


(0 


CO 


CO 


o 


CO 


0 


0 


0 


0 


0 


0 


CM 


0 


o 


o 


o 


o 


o 


o 


CO 


o 


CD 


CO 


T— 


CM 


T— 


CM 




M“ 












CO 




CO 


jO 


_co 


jO 


CO 


CO 




CO 












CO 




CO 


CO 


CO 


CO 


CO 


CO 


> 


'to 


> 


■> 


> 


> 


> 


> 


TD 


> 


TD 


TD 


■O 


-Q 


■O 


TD 


0 


TD 


0 


0 


0 


_0 


0 


0 


0 


0 


0 


0 


0 


*0 


0 


0 


O 


0 


O 


O 


o 


o 


o 


O 


c 


O 


C 


c 


c 


c 


c 


c 


0 


c 


0 


0 


0 


0 


0 


0 


o 


0 


o 


o 


o 


o 


o 


o 


T— 


o 


M- 


M" 


T— 




M- 


CO 


CM 

m 


m 

n 


o 


T3 


(D 


CM 


CM 


CM 

jr 



C\J 

CM 



Table 5 

Lesson Formats by Professional Development Staff-Teacher Role: Intermediate Level 



TJ 

© 



a 

a. 



TJ 

© 



a 

Q- 



O 

ERIC 



C C 
IO © 

o 

*£ 



O) C 

oo a 
I s - a 

11 m 
c £ 



CD 
k_ 

© 

-Q ««- C 

E <g • 
E oo o 
= II u 
Z C CD 

O) t 

o 



a) C 

o> © 
oo a 

11 m 
c £ 



o C 

v © 
o o 
11 ^ 
c £ 



© 

-Q o C 
P CM © 

= o> 

Z C « 

o> 5: 
o 



CM 

I ? £ 

z [L ® 
o> c 
o 



© 

3 5 8 

s ** 

O 



c 
0) 
co 
0 
Q O 

c 
© 

T 

3 

co 



o 
E 

CD 
TJ 

8 
d 
o 
c 



CO 




CO 


CO 


(0 








© 


CO 


© 


© 


© 


to 


CO 




CO 


© 


CO 


to 


CO 


© 

(0 


© 


CO 


CD 


CO 


© 


© 


© 


CO 


© 


o 


© 


a 


a 


a 


© 

o 


© 


CO 


T“* 


o 


CM 


T*“ 


co 


a 


© 

<J 


o 


CO 


oo 


CO 


h- 


__ 


M" 


00 


r* 


05 


c 


c 


c 


00 


co 


C 


c 


O 


o 


O 


c 


c 




o 


o 


TJ 


T 


T 


o 


O 


c 

o 


TJ 


TS 


© 


© 


© 


-0 


TJ 




© 


© 


CO 


CO 


CO 


© 

to 


© 


*o 


CO 


CO 


© 


© 


© 


CO 


© 


© 


© 


-Q 


_Q 


_Q 


© 

_Q 


© 


CO 


JOt 


-Q 


CO 


CO 


CO 


n 


© 

n 


co 

-4— ' 


© 


c 


-*-4 

c 


c 


CO 


© 


CO 


c 


c 


© 


© 


© 


c 


c 




© 


© 


o 


o 


o 


© 

8 


© 


c 


o 

v_ 


o 

v_ 


© 


v_ 

© 


v_ 

© 


o 

i— 


© 

o 


© 


© 


Q_ 


CL 


Q_ 


© 


© 


v_ 


CL 


Q_ 




hr 




CL 


CL 


© 

Q_ 






© 


© 


© 


. ■ 






© 


© 




-C 


-C 


© 


© 




-C 


-C 

-*w 


-♦w 

o 


O 


o 


-C 


-C 

-*w 


© 

-C 


O 


O 


o 


o 


o 


O 


O 


o 


o 


o 








o 


o 








TJ 


T 


T 






o 


TS 


TJ 


© 


© 


© 


-0 


T 




© 


© 


TJ 


TJ 


TJ 


© 


© 


T 


TJ 


TJ 


TJ 


TJ 


TJ 


-0 


*o 


© 


TJ 


TJ 


© 


© 


© 


-0 


*o 


*o 


© 


© 


© 


© 


iS 


© 


© 


*o 

© 

© 


© 


© 


*© 


© 


*© 


© 


© 


© 


© 


TJ 


T 


TJ 


© 


© 




T! 


T 


03 


03 


o: 


T 


TJ 


© 

-0 


03 


03 


C 


C 


c 


O 


o: 


a 


s 


C 


CO 


(O 


CO 


c 


c 


CO 


*C0 


CO 


(O 


CO 


(0 

to 


to 


,£ 


(0 


CO 


E 


E 


E 


E 


to 

co 



E E 

1= _C 



£ $ 
CO CO 
CD CD 
CO CO 
© © 
o o 

to CM 



£ £ E 

’5 § '5 £ 



to co 
CD CD 
CO CO 

S 8 

CM CO 
CO w 



co co Si ^ 



CO CO 

> > 



CO 

> 



o 
o 

CO GO T- 



-O t> 
0 © 

© © 
o o 
c c 
CD CD 

o 
o 



CO > 
CD 5 
CO CD 



8 

05 



CO 

■> 

TJ 

© 

© 

o 

c 

CD 

o 



CM 



co 

CM 



evidence of the use of Activity, Experimentation/ Investigation and Group Work across 
visits, whereas the percent of Seat Work decreased across visits. The percent of lessons 
that involved Teacher-directed activities tended to remain noteworthy across visits. 

Discussion 

The results of this study lend partial support for the presented program outcomes related 
to teachers’ instructional practices. More specifically, these data suggest that 
participating teachers are applying standards-based curricula in mathematics and science, 
using select best practices, and are attempting to use a variety of instructional strategies 
in their classrooms. Moreover, these data suggest that there is a change in the level of 
intervention that occurred across time during the implementation portion of the program. 
Although not intended to imply causality, this change in intervention level occurred 
concurrently with reports of growth in content knowledge by teachers and/or an increased 
comfort and confidence by participating teachers with pedagogical approaches supported 
during the program. These reports are in turn supported by data from additional internal 
studies that show gains in teacher knowledge as well as increased confidence in their 
ability to teach mathematics and science associated with participation in the program 
(Race, 2001). 

Present data also show that the curricula presented during classroom visits align with 
state learning standards in mathematics and science and that the implementation portion 
of the program also aligns with state learning standards in mathematics and science. 
Further investigation, however, suggests that the implementation portion of the program 
does not align or provide a “good-fit” with the instructional portion of the program 
relative to these learning standards. Such analyses should help program staff explore the 
objectives and outcomes of both the instructional and implementation portions of the 
program in their review of curriculum and to determine the level of alignment that is 
desired across these program components. Such decisions, however, are made more 
difficult in the absence of national (or state) benchmarks. For example, despite the fact 
that the National Council of Teachers of Mathematics recommends that a given 
mathematics standard should not receive equal emphasis across grade levels, NCTM has 
yet to provide a demarcation or benchmark as to the ideal apportionment of content 
specific standards across grade levels (NCTM, 2000). 

Although aligning curriculum with state learning standards is an important first step, past 
research suggests that more than alignment with standards is necessary to change 
teaching practices and improve student achievement (Elmore & Sykes, 1992; Elmore, 
1993; DeStefano & Prestine, 2000). To that end, the present findings suggest that there 
was frequent use of specific best practices and instructional strategies in classrooms by 
participating teachers across the implementation phase of the program at both the primary 
and intermediate level. Although it is difficult to causally tie these changes and 
variations in these data specifically to the program or the effects of the program per se, 
the overall picture that is presented is encouraging. It suggests evidence of the use of 
select best practices as well as the application of varied instructional strategies used over 
the time period of these visits. 
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Present data show that it was difficult to complete all visits as planned. Despite this 
difficulty, a substantial number of visits were successfully completed. A noted program- 
related reason for teacher attrition was the time commitment required to attend the 
instructional and implementation portions of the program that at times conflicted with 
other after-school programs or the perception that the program interfered with 
standardized test preparation. Based on self-report, some teachers did not feel the need 
for professional development or felt pressured to join the program. To some degree 
scheduling difficulties between the teacher and professional development staff led to this 
attrition. These program-related issues need and are being addressed by program staff. 

The qualitative data obtained from these implementation logs provide a descriptive 
context from which progress made by participating teachers can be assessed. Be that as it 
may, the implementation logs are not always used to their full capacity by all professional 
development staff. Even so, the nature of these data place limits on the full depth in 
which we can understand the complexity of the interactions between teachers and 
professional development staff, and provide an in-depth understanding of what is 
occurring in the classroom. Despite these limitations, the standardized log and the 
protocol that supports this process suggest the feasibility of this methodology. That is, 
present data support the use of this method to record hard-to-capture data from a 
relatively large group of teachers across an extended period of time as well as provide 
insights in understanding the mechanisms that underlie how the program may impact 
participating teachers. 

Planned research efforts will serve to link these implementation data with other data that 
we have collected from participating teachers such as their pre-program technology skills, 
content knowledge in mathematics and science as well as their attitudes toward using 
inquiry-based strategies, their reluctance to use traditional pedagogical approaches, use of 
computers in the classroom, and their confidence in teaching mathematics and science. 
Tying teachers’ practices in the classroom to these and other data sources, including 
students’ perspectives on activities and pedagogy in the classroom (Race & Powell, 

2000), can enhance our understanding of the target audience that this program is intended 
to serve as well as provide a clearer and more complete picture of program-related 
outcomes. 
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